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DETAILED ACTION 

Response to Amendment 

1 . The amendment received 1 2 February 2009, cancelling claims 9, 11, 12, 22, and 
24, and adding new claims 25-38, has been entered. Accordingly, claims 1-8, 10, 13- 
21, 23, and 25-38 are pending in the application. 

Response to Arguments 

2. Applicant has amended the title of the specification in response to the objection 
to the specification for the title being insufficiently descriptive. Accordingly, the objection 
to the specification on this basis is withdrawn. 

3. Applicant has amended the specification in response to the objection to the 
drawings for having reference numerals that did not correspond in the drawings and 
specification, by changing the step numbers to correspond to those step numbers 
presented in the drawings. Accordingly, the objection to the drawings on this basis is 
withdrawn. 

4. Applicant has amended claim 1 in response to the rejection under 35 U.S.C. 101 
as being drawn to nonstatutory subject matter, as the original claim 1 recited a method 
that was not tied to a particular apparatus and did not recite an eligible transformation. 
The amended claim, indicating that each step is performed "utilizing a computer", is tied 
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in a substantial way to the recited apparatus of a computer. Accordingly, the rejections 
of claims 1-8 and 10 under 35 U.S.C. 101 on this basis are withdrawn. 

5. The rejections of claims 10 and 12 under 35 U.S.C. 101 are rendered moot by 
the cancellation of those claims. 

6. With regard to claim 13 and its dependent claims 29-38, no rejection under 35 
U.S.C. 101 is being made at this time. Examiner construes the "computer readable 
storage medium" in accordance with the class of the examples provided in the 
specification (that is, not limiting it to the specific examples, but using the examples to 
imply a particular class of media); in particular, only physical computer readable media 
are mentioned in the specification, and Examiner therefore excludes from the scope of 
the claims' construction nonstatutory media such as signals, carrier waves, transmission 
lines, networks, etc. 

7. Applicant has amended claim 1 in response to the rejection under 35 U.S.C. 
102(b) as being anticipated by Park et al., "Segmentation and tracking of interacting 
human body parts under occlusion and shadowing," Proc. Workshop on Motion and 
Video Computing, pp. 105-1 1 1 (published 5 December 2002, hereinafter Park). 

In particular, amended claim 1 now includes the limitations previously recited in 
now-cancelled claims 9 and 1 1 , which were rejected under 35 U.S.C. 1 03(a) as being 
obvious over Park in view of Lo et al. (cited below), and Park in view of Sittler, R.W. 
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(cited below), respectively. However, as the scope of amended claim 1 is different from 
any claim rejected in the prior Office action, a new ground of rejection necessitated by 
the amendment is made below, and the arguments made by Applicant relative to claims 
1-8 and 10 are therefore moot. 

8. Applicant has amended claims 1 3 and 14 in similar fashion to claim 1 , so the 
same analysis given above for claim 1 applies to claims 1 3 and 1 4, as well as 
dependent claims 15-21 and 23. 

9. Applicant has argued concerning claim 7 that Magarey, J.F.A., U.S. Patent 
7,177,446 B2 (filed as application 10/207,140 on 30 July 2002, and published as U.S. 
Patent Application Publication 2003/0053661 A1 on 20 March 2003, hereinafter 
Magarey (Patent)) does not teach the limitations of claim 7 that are not disclosed by 
Park. While Examiner's prior explanation of how Magarey addresses the limitations of 
claim 7 was flawed, Magarey still teaches the limitations of claim 7 not disclosed by 
Park. A more detailed explanation of the pending rejection appears below. 

1 0. New rejections necessitated by the entry of new claims 25-38 are also made in 
this Office action. 

1 1 . Following are the objections and rejections pending in the current application. 
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Claim Rejections - 35 USC § 103 

12. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

13. Claims 1-4, 10, 13, 14-17, 23, 29-31, and 36 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Park et al., "Segmentation and tracking of interacting human 
body parts under occlusion and shadowing," Proc. Workshop on Motion and Video 
Computing, pp. 105-1 1 1 (published 5 December 2002, hereinafter Park), and further in 
view of Blackman et al., U.S. Patent 5,414,643 (issued 9 May 1995, hereinafter 
Blackman). 

14. With respect to claim 1 , Park discloses a method for tracking objects in a 
sequence of video images, comprising the steps of: 

storing object models relating to objects detected in previous video images of the 
sequence, the object models comprising values of characteristic features of the 
detected objects and variances of those values (inherent in the formation of blob- 
based object models and subsequent processing of those models for object 
tracking, section III); 

receiving a further video image of the sequence to be processed (inherent in the 
tracking of blobs throughout a video image, section III); 

detecting objects in the received video image ("A Gaussian mixture model is 
used to classify individual pixels.... Markov Random Field framework is used at 
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the blob level to merge the pixels.... The blobs are then grouped to form the 
meaningful body parts by a simple body model," section I); 

determining characteristic features of the detected objects (unary, binary, and 
tertiary features listed in section III.B); 

calculating a distance measure between each detected object and each object 
model on the basis of the respective characteristic features using a distance function 
which takes into account at least the variance of the characteristic features (Eq. 9, 10, 

11); 

matching the detected objects to the object models on the basis of the calculated 
distance measures (Fig. 3); and 

updating the object models using the characteristic features of the respective 
detected objects matched thereto (Fig. 5). 

Park does not explicitly disclose using a computer to perform the steps utilizing a 
computer, nor does Park disclose that, if a detected object is not matched to an object 
model then a new object model is stored corresponding to the detected object, and 

that the method further comprises counting, utilizing the computer, the number of 
consecutive video images for which each object is tracked, and outputting, utilizing the 
computer, a tracking signal indicating that tracking has occurred if an object is tracked 
for a predetermined number of consecutive frames. 

However, Park implies the use of a computer to perform the method (e.g., the 
authors work in the "Computer and Vision Research Center" at the "Department 
of Electrical and Computer Engineering, first page). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify the method of Park by performing the method on 
a computer, as implied by Park, motivated by the need to dramatically reduce the 
extreme length of time that performing such a method manually would entail. 

Furthermore, Blackman teaches storing a new object model if a detected object 
is not is not matched to an object model (table II shows that new track 11+ was 
added to the track list because it did not match a primary track, col. 7 lines 12-62). 
Blackman also teaches counting the number of consecutive video images for each 
object (col. 4 lines 12-22, indicating counting number of images out of the 
previous five consecutive images for each track) and outputting a track signal (the 
universal track list itself, Fig. 11) when the object is tracked for a predetermined 
number (four, col. 4 line 20) of consecutive frames. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify the modified method of Park by adding new 
tracked objects to the list if not matched to a previous tracked object and by outputting a 
track signal when a new object has been tracked for a sufficient number of consecutive 
frames, motivated by the need to track multiple independent objects and to eliminate 
noise and spurious detections. 

15. With respect to claim 2, Park and Blackman disclose the method according to 
claim 1, and Park further teaches that the distance measure is a scaled Euclidean 
distance (Eq. 9, 10, 11). 
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16. With respect to claim 3, Park and Blackman disclose the method according to 
claim 2, and Park further discloses that the distance function is of the form: 

for object model I and detected object k, where x N and yki are values of the characteristic 
features of a stored object model and a detected object respectively, On 2 is the 
corresponding component of the variance of each feature, and the index i runs through 
N features of an object model (mathematically equivalent to Eq. 11). 

17. With respect to claim 4, Park and Blackman disclose the method according to 
claim 1, and Park further discloses that the distance measure is the Mahalanobis 
distance (Eq. 11). 

18. With respect to claim 10, Park and Blackman disclose the method according to 
claim 1 . Blackman further discloses deleting an object model when the object has not 
been tracked for a number of consecutive frames (col. 7 lines 26-29 and Table III). 

1 9. Claims 1 3, 29-31 , and 36 are rejected for the same reasons given above for the 
corresponding method in claims 1-4 and 10 above, respectively. 
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20. Claims 14-17 and 23 are rejected for the same reasons given above for the 
corresponding method in claims 1-4 and 10 above, respectively. 

21 . Claims 5, 1 8, and 32 are rejected under 35 U.S.C. 1 03(a) as being unpatentable 
over Park and Blackman as applied to claims 1,14, and 13 above, respectively, and 
further in view of Marcenaro et al., "Multiple Object Tracking under Heavy Occlusions by 
Using Kalman Filters Based on Shape Matching," Proc. 2002 Int'l. Conf. on Image 
Processing, Vol. 2 pp. 341-344 (published 22 September 2002, hereinafter Marcenaro). 

22. With respect to claim 5, Park and Blackman disclose the method of claim 1 , but 
do not disclose predicting the values of characteristic features of object models for the 
received frame. 

However, Marcenaro discloses using a Kalman filter, which predicts the values of 
characteristic features and then uses those feature predictions to update the object 
models (section 4). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to combine the invention of Park and Blackman with the 
application of the Kalman filter as disclosed by Marcenaro, motivated by the reduction in 
sensitivity to noise that the Kalman filter provides, a property of Kalman filters that is 
well known in the art.. 

23. Claims 18 and 32 are each rejected for the same reasons given above for the 
corresponding method in claim 5. 
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24. Claims 6, 19, and 33 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Park and Blackman as applied to claims 1, 14, and 13 above, respectively, and 
further in view of Harville, U.S. Patent 7,003,136 B1 (filed as application 10/133,151 on 
26 April 2002, hereinafter Harville). 

25. With respect to claim 6, Park and Blackman disclose the limitations of claim 1 , 
but do not disclose increasing the variances of characteristic feature values when an 
object model is not matched to a detected object. 

However, Harville discloses incrementing the state variances when no detected 
object is matched to an object model (col. 14, lines 17-22). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to combine the invention of Park and Blackman with the 
increase in state variances disclosed by Harville, motivated by the need to decrease the 
level of tracking confidence for the person (Harville, col. 14 lines 20-21) and to 
accommodate a larger valid area for reacquiring the tracked object. 

26. Claims 19 and 33 are each rejected for the same reasons given above for the 
corresponding method in claim 6. 

27. Claims 7, 20, and 34 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Park and Blackman as applied to claims 1,14, and 13 above, respectively, and 
further in view of Magarey, U.S. Patent 7,177,446 B2 (filed as application 10/207,140 on 
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30 July 2002 and published as U.S. Patent Application Publication 2003/0053661 A1 on 
20 March 2003, hereinafter Magarey (Patent)). 

28. With respect to claim 7, Park and Blackman disclose the limitations of claim 1 , 
but do not disclose updating the characteristic feature values with the average values 
found over a predetermined number of previous images whenever the object model is 
not matched with a detected object. 

However, Magarey (Patent) discloses updating reference data with the average 
of feature vectors of prior frames when determining that the track of the feature has 
been lost. When track is lost, the state is updated with the predicted state vector (col. 9 
lines 46-54). The last two elements of this vector, the velocity (corresponding to the 
characteristic feature values), is calculated in Eq. 12. As can be seen from matrix D 
(Eq. 3), the bottom two rows of the output are equal to the velocity at the previous 
frame. Therefore, the characteristic feature values (velocity) are updated with the 
average values found over one previous image whenever the object model is not 
matched with a detected object (i.e., tracking is lost). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to combine the invention of Park and Blackman with the 
average value update of Magarey (Patent), motivated by the need to update the tracked 
object with a feature vector representative of its past behavior in order increase the 
likelihood of reacquiring tracking at a later time. 
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29. Claims 20 and 34 are each rejected for the same reasons given above for the 
corresponding method in claim 7. 

30. Claims 8, 21 , and 35 are rejected under 35 U.S.C. 1 03(a) as being unpatentable 
over Park and Blackman as applied to claims 1,14, and 13 above, respectively, and 
further in view of Senior et al., "Appearance Models for Occlusion Handling", Proc. 2 nd 
IEEE Int'l. Workshop on Performance Valuation of Tracking and Surveillance, pp. 108- 
1 1 5 (published 9 December 2001 , hereinafter Senior). 

31 . With respect to claim 8, Park and Blackman disclose the limitations of claim 1 , 
but do not disclose considering an object to be occluded if an overlap with another 
object is detected. 

However, Senior discloses the determination of occlusion when objects overlap 
(Fig. 4, 5, and section 6), and further describes the resolution of such occlusion 
determinations. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to combine the invention of Park and Blackman with the 
occlusion resolution of Senior, motivated by the increased accuracy of tracking when 
the tracking process can distinguish at least partially from among overlapping objects, a 
necessity when dealing with real world scenarios (Senior, abstract and section 1).. 

32. Claims 21 and 35 are each rejected for the same reasons given above for the 
corresponding method in claim 8. 
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33. Claims 25, 27, and 37 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Park and Blackman as applied to claims 1,14, and 13 above, 
respectively, and further in view of Waters et al., U.S. Patent 6,256,046 B1 (issued 3 
July 2001 , hereinafter Waters). 

34. With respect to claim 25, Park and Blackman disclose the method according to 
claim 1 . Park further discloses that the characteristic features comprise features 
describing the shape (border pixel set, Sec. III.B.) and color (mean intensities of 
HSV color components, Sec. III.B.) of each detected object. 

Neither Park nor Blackman disclose that the characteristic features comprise the 
velocity of each detected object. 

However, Waters teaches tracking the velocity of a blob object (col. 4 line 65 
through col. 5 line 13). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify the invention of Park and Blackman by including 
the velocity of the object in the characteristic features, as taught by Waters, motivated 
by the ability to discern, for example, "intent" of the represented object (Waters, col. 5 
lines 10-13). 

35. Claims 27 and 37 are each rejected for the same reasons given above for the 
corresponding method in claim 25. 
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36. Claims 26, 28, and 38 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Park, Blackman, and Waters as applied to claims 25, 27, and 37 
above, respectively, and further in view of Zhou et al., "Tracking and classifying moving 
objects from video", Proc. 2 nd Int'l Workshop on Performance Evaluation of Tracking 
and Surveillance (published during or before December 2001 , hereinafter Zhou) and 
Durnell, L, International Patent Application Publication WO 03/017203 A1 (published 27 
February 2003, hereinafter Durnell). 

37. With respect to claim 26, Park, Blackman, and Waters disclose the method of 
claim 25, wherein the characteristic features comprise the velocity (Waters, col. 4 line 
65 through col. 5 line 13), size (number of pixels in the blob, Park, Sec. III.B.), and 
dominant color representation (mean intensities of HSV color components, Park, 
Sec. III.B.). 

None of Park, Blackman, and Waters teaches the inclusion of the major axis to 
minor axis ratio of the best-fitting ellipse or the orientation of the major axis of the 
ellipse. 

However, Zhou teaches tracking the orientation of the major axis of the ellipse 
("Object orientation is defined as the angle between the major axis and the 
horizontal axis", Sec. 3), and Durnell teaches tracking the aspect ratio, or the ratio of 
the minor axis to the major axis (p. 6; this is simply the reciprocal of the ratio calculated 
in the present application, and would have been obviously interchangeable to an 
ordinary artisan at the time of the invention). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify the invention of Park, Blackman, and Waters by 
including the ellipse orientation and aspect ratio in the characteristic features, 
representing the application of a known technique to a known method to obtain 
predictable results. 

38. Claims 28 and 38 are each rejected for the same reasons given above for the 
corresponding method in claim 26. 

Conclusion 

39. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Auty etal., U.S. Patent 5,809,161 . 
Abbott et al., U.S. Patent 5,999,634. 

Collins et al. "Algorithms for cooperative multisensor surveillance." Proc. IEEE, 
Vol. 89 No. 10, pp. 1456-1477. 

40. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See M PEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Barry Drennan whose telephone number is 571-270- 
7262. The examiner can normally be reached on Monday through Thursday, 9am to 
5pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on 571-272-7415. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Barry Drennan/ 
Examiner, Art Unit 2624 



A/ikkram Bali/ 



Supervisory Patent Examiner, Art Unit 2624 



